Integrating Articulatory Features into Acoustic Models for Speech Recognition

It is often assumed that acoustic-phonetic or articulatory features can be beneficial for automatic speech recognition (ASR), e.g. because of their supposedly greater noise robustness or because they provide a more convenient interface to higher-level components of ASR systems such as pronunciation modeling. However, the success of these features when used as an alternative to standard acoustic speech signal representations (e.g. MFCCs) has so far been demonstrated only for limited domains, such as phone recognition or smallvocabulary speech recognition. On more challenging tasks, e.g. large-vocabulary speech recognition, standard acoustic features have consistently shown a superior performance. This study compares the performance of standard acoustics-based systems to that of articulatory feature-based systems on medium to large vocabulary recognition tasks. Results suggest that, for an optimal recognition performance, it is more advantageous to selectively combine information from both acoustic and articulatory representations than it is to use an articulatory feature-based representation alone. Data-driven techniques are applied to determine what kind of information articulatory features can contribute in addition to standard acoustic speech features.